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Instructional Video Evaluation 

The advisory committee for the High Plains Intel-mountain Center for 
Agricultural Heath and Safety (HI-CAHS) strongly recommended videos 
be used in the education programs developed by the HI-CAHS Education 
and Training Program. In keeping with this mandate, HI-CAHS has identified several videos 
that are being considered for training programs. A major problem arose in how to evaluate 
these videos for instructional content, thus the need for this research. 

The Need for Instructional Video Evaluation 

For this project, videos are considered "products." Product evaluation methods are 
needed to help users and producers arrive at better decisions based op reliable, accurate, and 
complete information (Krink & Gustafson, 1986). From idea inception through final product 
and beyond, evaluation needs to be incorporated into the production and viewing process. 
Formative and summative evaluation should H have a place all through the production process: 
before, during and after- (Hausman, 1990, p. 124). Evaluation is needed to provide the 
industry with more direction about what constitutes effective, high-quality educational 
products: 

Until evaluation becomes an integral part of the video design and production process, there will be 
no experienced-based knowledge of what video approaches work, with what type of audiences, 
under what kinds of conditions, and in what type of content areas. (Sneed, 1991, p. 8) 

Evaluation of products, including formative and summative procedures, is an important 
aspect of production, with the aim of "making administrative decisions and decisions about 
course improvement M (Krink & Gustafson, 1986, p. 217). 

The lack of quality evaluation techniques, the emphasize on the negative, and too 
many variables to measure are barriers to evaluation reaching its intended goals. The lack of 
quality evaluation techniques is partly responsible for the misconception about what 
evaluation is and how to achieve it. In the real world, it is impossible to "isolate the effects 
of a video program on a particular outcome, like increased sales or improved job 
performance" (Sneed, 1991, p. 5). From its formative and summative roots in the design 
and production process, evaluation can be accomplished by the organizer of the learning 
event and the end user to decide whether a tape is suitable for their particular instructional 
program. The instructional video product "should be a realization that there is a need for 
moving visual material of this type in a particular instructional situation either in a supporting 
role or as the main vehicle of instruction and that no suitable material is already available" 
(Ellington, 1985, p. 176). 

Instruction al Video Evaluation Imminent 

Compiling the suggestions for quality indicators for videos found in books, articles, 
and forms by various authors (Dube, 1980; Ellington, 1985; Handbook of Forms . 1985; 
Hunter, 1990; Hutton, 1984; Krink & Gustafson, 1986; National Career Development 
Association [NCDA], 1992; Pett, 1989) and synthesizing the instructional design 
methodology of Brookfield (1985, 1986), Friere (1970), Galbraith (1991, 1992), and Seels & 
Glasgow (1990), an instrument was developed to aid the reviewer in evaluating a video for 
its instructional quality (see Appendix A). The following is a list of the quality indicators 
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used in the instrument, with detailed information concerning why it was included and how to 
judge the video for each indicator. 



Instructional Video Quality Indicator Descriptions 

Content 

The content of the video is a prime concern in an instructional setting. The video 
must be accurate, useful, and free from bias. 

Quality in dicator #1: Accuracy . If the content of the video is not correct and up-to- 
date then the video is not ideally useable for learning. The content must be accurate and 
current (NCDA, 1992). The video must portray situations that are current and useful in 
today's world. There may be portions that are usable and portions that should not be 
included. The criteria for establishing what should not be included is dependent on what 
impact the inaccurate information will have on the learner. 

Quality indi cator #2: Usefulness . The content of the video must be generally useful. 
The video should stimulate, motivate and inform the learner to act on the information that is 
being presented. Ideally, learners should consider and/or incorporate the ideas presented 
(Krink & Gustafson, 1986; NCDA, 1992). 

Quality indi cator #3: Bias-Free . The video should be bias-free, including 
stereotyping because of age, gender, ethnicity, race, physical impairment, values, dress, 
language, or social class (Krink & Gustafson, 1986; NCDA, 1992). If the video is not free 
from bias, the educational objectives may be greatly effected or compromised. Individuals 
depicted in the video should not be shown as a role stereotype for the task being enacted or 
illustrated. "A video lacking a progressive social orientation would also be deficient in 
objectivity and accuracy of information" (NCDA, 1992, p. 6). 

Instructional Plan 

Instructional design models are used to control the design process. These models 
generically include five steps: analysis, design, development, implementation, and evaluation. 
During the analysis step the instructional designer might perform a needs assessment and 
create a problem statement. The design entails creating a plan of operation that would guide 
the designer in setting competencies and outcomes, writing objectives, creating assessment 
strategies and a selection of the proper media (videotapes, texts, facilitation aids, etc.). 
Development means turning that plan into reality, creating the necessary session plans, study 
guides, workbooks, job aids, etc. that are needed for delivering the instructional program. 
When the program is ready it is implemented on a trail basis and evaluated so improvements 
can be made (Seals & Glasgow, 1990). 

The concern with the selection of the proper video to use in a learning activity is with 
this design phase. During the design, a plan must be established that results in the learners' 
needs being accomplished through the use of the video. This plan can be generically 
outlined as having an introduction, a body, and closure. The introduction should include the 
objectives of the session, benefits that will be derived from the session and some sort of 
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"attention getter." The main body should have a presentation of the content, some 
demonstration or application of this content. It should also allow time for learner reflection 
on the content and application. Continue this, presentation-*application-»reflection cycle, 
until all the objectives expressed in the introduction are met (Brookfield, 1985, 1986; Friere, 
1970; Galbraith, 1991, 1992). Closure should review what has been learned during the 
session and motivate the learner to apply the content to their lives. The following quality 
indicators are organized around this structure and are considered important when evaluating a 
video's worth for instructional purposes. 

Quality indicator Hi St ate the objectives . It is important to begin the video with a 
motivating introduction to stimulate interest and to meet the expectations of the learner. 
People remember the first things presented in a program, so it is important to include the key 
learning elements in the introduction (Pett, 1989; Krink & Gustafson, 1986; NCDA, 1992). 
The introduction should emphasize a description of the objectives or key elements contained 
in the video. 

Quality indicator #5: C ontent presentation . The content should be controlled to 
promote understanding. The video should simplify complex tasks and avoid introducing 
extraneous information. It should not attempt to introduce too much material or introduce 
too much detail. Do not clutter the scene (Dube, 1980; Krink & Gustafson, 1986). "Divide 
complex ideas into simple ones and avoid introducing extraneous information, unwanted 
sounds, or inappropriate pauses" (Pett, 1989, p. 2). "The very nature of video programming 
is superficial and simplistic" (Hunter, 1986, p. 20). "If you try to cover too much material 
or introduce too much detail, some of your audience may become confused or 'get lost'" 
(Ellington, 1985, p. 175). 

Quality indicator 16: L earner application . The video should suggest methods for 
learner application of the newly acquired knowledge. Recommendations for practice of what 
is being discussed should be specified. Practice can be designed into the general program 
design or into the video itself (Dube, 1980; Pett, 1989). "A collaborative and critically 
reflective learning experience must be a combination of contemplation and action" (Galbraith, 
1991, p. 4). 

Quality indicator HHx L earner reflection . The video should suggest methods for 
reflection on the content and application. Reflection, silence, or time should be allowed for 
the learner to react to a scene or statement (Dube, 1980; Pett, 1989). It is also important for 
the facilitator to interact with the learner to provide response to the learner's application of 
the material. There should be "alternating and continuous engagements by teachers and 
learners in exploration, action, and reflection" (Brookfield, 1986, p. 15). 

Quality indicator #8; Meet the objectives . The video should meet the learning 
objectives and needs of the learner. What is being visually depicted should fit the learning 
objectives (Dube, 1980; Krink { Gustafson, 1986; National Career Development Association 
[NCDA], 1992). As in the introduction, people also remember the last things presented in a 
program, therefore, it is important to have the key learning elements in the summary or 
conclusion (Pett, 1989). 

Quality indicator #9: L earner Interaction . The video should be conducive to 
learner interaction. Videos can often be used to promote active learning. Video-student 
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interaction can be designed into the overall program design. The video does not need to be a 
one-way communication, screen to the student; it can be produced to interact with the learner 
(Dube, 1980; Hunter, 1986; Krink & Gustafson, 1986). 

Quality indicator #10; Integration into the learning environment . Videos can be 
used to add emphasis and supplement more traditional methods. The video is an excellent 
media for bringing Experiences and places into the classroom" (Dube, 1980, p. 28). 

Technical Production 

The following quality indicators are derived from good design practices of the 
producers of video products for instruction. The video industry is charged with producing 
materials that give quality methods to the instructional process. Characteristics that are 
inherent to the medium are: (1) products foster unification and involvement between the 
viewer and the subject matter of the video; (2) video viewing provides one-way 
communication along with transcending space and time; (3) the viewer is enveloped with 
sound along with visual perspectives; (4) video viewing involves all of the senses 
simultaneously; and (5), video demands participation from the viewer (Dube, 1980). Hunter 
(1990) identifies specific characteristics of a quality video product as transcendence, attention 
manipulation, detail, special effects, economy, independence and interdependence. 

Quality indicator #11; General video design characteristics . Videos are conducive 
to providing well planned, structured, and organized instructional design (Dube, 1980; 
Ellington, 1985; Krink & Gustafson, 1986; NCDA, 1992; Pett, 1989). The video should not 
appear to be a video; it should appear real. The technology should become transparent and 
non threatening to the learner (Hutton, 1984). Transcendence refers to a video's ability to 
move back and forth and in and out of time and space. The video camera can enter a world 
that the learner cannot and allows the learner to "confront otherwise nebulous theories" 
(Hunter, 1990, p. 18). The camera can go where the learner cannot, however, care must be 
taken to prevent presenting a false idea of reality (Hunter, 1990). 

Quality indicator #12; Focus on the intended content . The video should avoid 
content not related to the subject matter stated in the introduction or implied in the title. 
Digressions could lead to confusion and would be a waste of video time. The video should 
stay focused on the content and logically flow (Ellington, 1985; NCDA, 1992). In designing 
a learning program, it is important that the main points be repeated several times so that they 
are not missed (Krink & Gustafson, 1986; Pett, 1989). Accomplish this through examples, 
illustrations, and questions (Krink & Gustafson, 1986). "The length of the video should be 
weighed in balance with the content, intended audience" (NCDA, 1992, p. 4) and 
instructional value. 

Quality indicator 113; Visual quality . Make the angle at which the camera is 
viewing the scene or training environment appropriate to what is being learned. Is the 
camera looking at the scene from the learner's point of view, in situations where the learner 
would benefit from that point of view? This is especially important when psychomotor skills 
are being taught (Dube, 1986). 

Was the speed suitable for the audience or subject matter? Some learners prefer the 
scenes to change quickly, while others prefer a slower pace (Dube, 1986). Varying types of 
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camera shots, close-ups to long shots, also provide variety in the video (Ellington, 1985; 
Pett, 1989). 

There are a variety of special electronic procedures and effects that can alter a video 
beyond presenting just what a camera records. These effects include: M pop-on arrows, 
dissolves that intimate time distortion, slow and fast motion, animation, art work, split 
screen, captions and cartoon characters" (Hunter, 1990, p. 19). Special effects accomplish a 
variety of things. They add impact, meaning and brilliance to instructional material along 
with allowing for concrete topics, to be presented side-by-side for comparison and contrast. 
Hunter goes on to suggest that special effects enhance a learner's processing skills and 
improve video products. Using special effects can prompt learners to give unusual attention 
to specific details, ask questions and draw conclusions from instructional materials. 
Graphics, animation and special electronic transitions between scenes help learners 
understand video materials and conceptualize otherwise unclear concepts. 

Quality indicator 114: Au dio quality . The narration should consist of words that 
are familiar to the audience. It should be informal, suitable, easily understood and free of 
stereotyping (Krink & Gustafson, 1986; NCDA, 1992; Pett, 1989). Although there may not 
be any difference in learning with respect to the speed of the narration, it is better to err on 
the slow side (Dube, 1980). In any case, the speed should be appropriate for the subject 
matter (Dube, 1980; Krink & Gustafson, 1986; Pett, 1989). 

The background music should be fitting for the visual affect or audio narration. 
"Music may distract the students from what they are trying to learn" (Dube, 1980, p. 25). 
Use music appropriately for affective learning (Pett, 1989; NCDA, 1992). Use of 
background noises can be conducive to learning, such as farm noises in an agricultural safety 
video. Sound effects will add emphasis to the visual tract of a video to enhance learning 
(Pett, 1989; NCDA, 1992). 

Quality indicator 015: Au dio-visual relationship . A well-combined audio-visual 
program is best for learning. It is good to name items that are being seen in the visual tract. 
Visually showing the audio's key words and announcing the visual's main points places 
emphasis where it needs to be, in key areas (Dube, 1980; Pett, 1989). Remember, videos 
are essentially a visual media, thus the audio track plays a supportive role (Ellington, 1985). 
The audio and visual component should never contradict each other and should flow together. 
The two modes should compliment each other (Pett, 1989), having a variety of sounds and 
visuals attract and hold attention (Pett, 1989). 

Supplement al Materials 

The information that accompanies a video is important in the instructional design of 
the product. It must be accurate and useful to the learner and the facilitator. It must state 
the purpose of the video, give a summary of the content of the video, clarify any terms or 
procedures that may not be clear from the video, and provide the leader and facilitator with 
a guide to using the video (NCDA, 1992). 

Video is not a "magic bullet" (Hart, 1984, p. 87). For the product to be of high 
quality and effective, a program can require considerable facilitation and hard-copy support 
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materials. "Programs that achieve the most successful educational results are known to have 
supplemental materials that correspond to the [video] telecast" (Hunter, 1990, p. 20). 

Quality indicator 016: Provide introductory Information . Before watching the 
video, the learner should be aware of the purpose and objectives of the video. The video 
should accomplish what is stated in the supplemental materials and shared with the learner. 
The credit information (date of production, rev'iion, producer, title, etc.) in the video should 
match what is in the supplemental materials (NCDA, 1992). 

Quality indicator 11 7i Clarify and summarize content . This may consist of 
providing job aids or diagrams that help in understanding the material. It may mean defining 
terms and giving other sources for further investigation. The summary should be useful in 
understanding what the video contains and should match what is on the medium. The 
supplemental materials should also contain aids in following the video for learning, such as 
scripts, reproductions of important content, and stop-and-discuss points. There should be 
suggested activities in the materials to aid in understanding, such as discussion questions, 
role plays or simulation exercises (NCDA, 1992). 



How to Use the Instructional Video Evaluation Instrument 

The main purpose of the evaluation instrument is to allow reviewers to establish 
baseline information related to a video before a decision is made to recommend the product 
for inclusion in a learning event. The Instrument can be copied and distributed to several 
subject matter experts, instructors and/or end users for independent review of the video. 
Compile the results (see the Table 1 below for an example and Table 2 is a blank for your 
convenience). Each quality indicator is normally weighted the same, however, an evaluation 
coordinator could adjust the weight of one or more indicators to add emphasis. With this 
information, the value of the video for instructional purposes can then be quantified. 
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Table 1 - Sample Comp ilation of Data D erived from the Instrument 



Quality 
Indicators 


Reviewer 
One 


Reviewer 
Two 


Reviewer 
Three 


Total 
(Mean) 


1. Accurate 


5 


4 


4 


13(4.3) 


2. Useful 


4 


4 


3 


11(3.7) 


3. Bias-Free 


3 


4 


4 


11(3.7) 


Content Sub-Total 


12(4.0) 


12(4.0) 


11(3.7) 


35(3.9) 


4. Objectives 


4 


5 


4 


13(4.3) 


5. Presentation 


3 


4 


5 


12(4.0) 


6. Application 


3 


3 


3 


9(3.0) 


/. Keiiection 


3 


2 


3 


8(2.7) 


o. Met UDjectives 


4 


5 


3 


12(4.0) 


y. interaction 


3 


A 

4 


3 


10(3.3) 


iu. integration 


4 


A 

4 


A 

4 


12(4.0) 


instructional oUD-iotai 


24(3.4) 


27(3.8) 


25(3.5) 


mm /*% >»\ 

76(3.6) 


1 1 . uesign 


C 

J 


5 


5 


15(5.0) 


iz. rocuseo 


4 


5 


4 


13(4.3) 


u. visual 


3 


4 


A 

4 


11(3.7) 


14. Audio 


4 


4 
*t 


4 


iz^**.uj 


15. Audio-Visual 


4 


3 


4 


11(3.7) 


Technical Sub-Total 


20(4.0) 


21(4.2) 


21(4.2) 


62(4.1) 


16. Intro Info 


4 


5 


4 


13(4.3) 


17. Summary 


3 


3 


3 


9(3.0) 


Supplemental Sub-Total 


7(3.5) 


8(4.0) 


7(3.5) 


22(3.7) 


Quality Total(Mean) 


63(3.7) 


68(4.0) 


64(3.8) 


195(3.8) 
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Table 2 - Blank Compilation Table 



Quality 

Indicators 


Reviewer 


Reviewer 

Tu/n 
l WU 


Reviewer 
l nree 


Total 
^ivieanj 


1 Accurate 










2 Useful 










% Rias-Frpp 










Content Snh-Tntal 










» • V/I/JvvUtI/i) 










S Presentation 










6. Aonli cation 










7. Reflection 










8. Met Obiectives 










9. Interaction 











10. Integration 










Instructional Sub-Total 










11. Design 










12. Focused 










13. Visual 











14. Audio 










15. Audio- Visual 










Technical Sub-Total 










16. Intro Info 










17. Summary 










Supplemental Sub-Total 










Quality Total(Mean) 











Copyright • 1993 Permiition is hereby granted to cod u*cn to duplicate thii Uble for uae in evaluating video* with appropriate credit given to High Plaini 
Intermountain Center for Agricultural Health and Safety (HI-CAHS), Education A Training Program, Bart Beaudin, Ph.D., Team Leader, Colorado SUte 
Univeriity, Fort Collini, Colorado. 
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